Adaptive model-based system to automatically quantify fall risk

ABSTRACT

A method of conducting a structured test with a user. The structured test includes presenting an instruction to the user to perform at least one action for a test duration to test a current state of the user. Sensor data is received from one or more sensors worn by the user or placed near the user that indicate at least a motion or a posture of the user. The motion or the posture of the user during the structured test is measured with the sensor data. At least one of a timing of providing an instruction to perform at least one action and a test duration of a subsequent structured test is adjusted based on the measured motion or posture of the user, and the subsequent structured test is conducted with the user based on the adjusted at least one of the timing and the test duration.

FIELD OF ART

This disclosure relates generally to assessing the fall risk of patientsand particularly to an adaptive system to automatically quantify fallrisk.

BACKGROUND

Injuries caused by falling impact many people today, especially theelderly. The resulting hospitalizations are responsible for asignificant resource cost to health care systems. Though there arefactors tied to a greater risk of falls, these factors are not easilymeasured and assessed for each individual. Further, fall tracking is notcommonplace, and most falls are self-reported by the patient to aphysician, if reported at all. Individuals with neurological disorderssuch as Parkinson's disease and multiple sclerosis have a greater riskof falling because symptoms of these diseases have a toll on heircentral nervous systems and motor systems.

Existing systems can detect when an individual falls, and seekassistance for the individual. However, these systems are reactivemeasures, and a proactive solution currently does not exist. Further,existing systems do not generate recommendations regarding how anindividual can reduce their risk of falling. Not only is it important todetect the fall, but also to monitor an individual's behavior on aregular basis to assess their fall risk over time and predict when theymay be more vulnerable to falling. Since many elderly individuals liveon their own, it is important for others (e.g., caretaker and familymembers) to have a way to remotely monitor the well-being of theindividual and respond with appropriate help if the individual hasfallen, which current systems do not provide.

SUMMARY

A system is provided to determine fall risk including an adaptive modelfor automatically predicting the fall risk of a user. The system trainsa model using machine learning techniques and various featuresincluding, but not limited to, user inputs (e.g., age, vision, use ofassistive devices in walking, etc.), sensor data (e.g., data collectedfrom sensors associated with the user), external source data (e.g.,population data about fall risk of a population of users), and in somecases structured user tests (e.g., tests performed by the useroccasionally over time to assess sway radius and other conditions of theuser on that day). Sensor data can be recorded from depth sensors,microphones, depth sensors, and a variety of wearable sensors on theuser. The wearable sensors can be in the form of a wristband,wristwatch, pendant of a necklace, or belt clip. Data may be uploadedand synchronized to a cloud server where the fall risk model is updated.A notification or other feedback generated from the system can be sentto the user, the user's caregivers and other health care providers, theuser's family and friends, and the like. The notification or feedbackmay include the user's fall risk as well as recommendations regardinghow the user can reduce the risk of falling.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a computing environment of an adaptivesystem to automatically quantify fall risk, according to one embodiment.

FIG. 2 is a block diagram of a fall risk module of an adaptive system toautomatically quantify fall risk, according to one embodiment.

FIG. 3A is a chart illustrating the trace of a healthy user with a fallrisk near average as measured by the adaptive system to automaticallyquantify fall risk, according to one embodiment.

FIG. 3B is a chart illustrating the trace of an unhealthy user with afall risk greater than average as measured by the adaptive system toautomatically quantify fall risk, according to one embodiment.

FIG. 4A is a diagram illustrating the posture and center of gravity of ahealthy user with a fall risk near average as measured by the adaptivesystem to automatically quantify fall, risk, according to oneembodiment.

FIG. 4B is a diagram illustrating the posture and center of gravity ofan unhealthy user with a fall risk greater than average as measured bythe adaptive system to automatically quantify fall risk, according toone embodiment.

FIG. 5A shows a diagram illustrating timelines with fall events of auser, according to one embodiment.

FIG. 5B shows a diagram illustrating a tensor of data used by a fallrisk module, according to one embodiment.

FIG. 5C shows a data flow diagram illustrating the interactions betweenvarious types of data used to determine a fall risk score, according toone embodiment.

FIG. 6 is a flowchart of the process of periodically retraining a modelfor determining a fall risk score for a user, according to oneembodiment.

The figures depict various embodiments of the invention for purposes ofillustration only. One skilled in the art will readily recognize fromthe following discussion that alternative embodiments of the structuresand methods illustrated herein may be employed without departing fromthe principles of the invention described herein.

DETAILED DESCRIPTION System Overview

FIG. 1 is a block diagram of a computing environment 100 of an adaptivesystem to automatically quantify fall risk according to one embodiment.The embodiment illustrated in FIG. 1 includes a fall risk module 110, aclient device 130, wearable sensors 140 (e.g., 140 a and 140 b), a depthsensor 150, and an external source 160 connected to each other by anetwork 120. Embodiments of the computing environment 100 can have onlyone wearable sensor 140. Embodiments of the computing environment 100can have multiple client devices 130, depth sensors 150, and externalsources 160 connected to the network 120. Likewise, the functionsperformed by the various entities of FIG. 1 may differ in differentembodiments.

A client device 130 is an electronic device used by a user to performfunctions such as entering user data and events as input to the fallrisk module 110 to train a model to quantify the user's fall risk. Forexample, the client device 110 may be a mobile device, a tablet, anotebook, a desktop computer, or a portable computer. The client device110 includes interfaces with a display device on which the user may viewwebpages, videos and other content. In addition, the client device 110provides a user interface (UI), such as physical and/or on-screenbuttons with which the user may interact with the client device 110.

The wearable sensor 140 is an electronic device used to collect datafrom the user and wirelessly transmit the collected data over thenetwork 120 to the fall risk module 110. Embodiments of the wearablesensor 140 are sensors including, but not limited to, inertialmeasurement units (IMU), accelerometers, gyroscopes, magnetometers, andphoto diodes. Embodiments of the wearable sensor 140 may have formfactors such as a wristband, wristwatch, pendant of a necklace, or beltclip, walking cane, and the like. The user can also have more than onesensor, such as one wristband and one walking cane sensor. Wearablesensors 140 can also track vital signs or biological or biometric data,including heart rate, breathing, temperature, blood pressure, etc.

The depth sensor 150 is an electronic device used to collect depth datafrom the user and wirelessly transmit the collected data over thenetwork 120 to the fall risk module 110. Unlike the wearable sensor 140,the depth sensor 150 is located in a stationary position in the vicinityof or near the user. For example, the depth sensor 150 can be positionednear a television in the user's home. The depth sensor 150 (e.g., acamera) may record video, record audio, take photos, capture other mediacontent about the user and the user's environment, and other forms ofdepth data. For example, this sensor 150 can be used to record theuser's gait as the user walks, the user's standing posture, lightingconditions in the user's room, compliance by the user withdoctor-prescribed walking assistance device (canes, walkers, etc.),daily activity types by the user, daily activity duration, among otherinformation. Other types of sensors can also be included in the room,such as temperature sensors to determine temperature in the room, lightsensors to detect room lighting conditions, etc. There can be a sensor150 in multiple rooms of the user's home.

The network 120 enables communications among network entities such asthe fall risk module 110, client device 130, wearable sensors 140, depthsensor 150, and external source 160. In one embodiment, the network 120includes the Internet and uses standard communications technologiesand/or protocols, e.g., BLUETOOTH®, WiFi, ZIGBEE®, clouding computing,other air to air, wire to air networks, and mesh network protocols toclient devices, gateways, and access points. In another embodiment, thenetwork entities can use custom and/or dedicated data communicationstechnologies.

The external source 160 provides information to the fall risk module 110to train the model to quantify the user's fall risk. For example, theinformation can include data from a population of both healthy andunhealthy users, as well as medical practice standards (e.g.,prescribing guidelines of consensus practice recommendations forreducing fall risk). In some embodiments, the information provided bythe external source 160 is collected each time a fall risk score isdetermined by the fall risk module 110. In other embodiments, the fallrisk module 110 builds up one or more of its own databases (e.g., seeFIG. 2) of information about the population of users either in advanceor as fall risk scores are determined such that the fall risk module 110can utilize its own source of information about the user population.Such a fall risk module database can be updated regularly to ensure themost accurate and up-to-date information is kept on hand.

The external source 160 may also include historical health data of auser (e.g., a user's electronic medical records, or EMRs) from varioushealth record sources (e.g., hospital records, records at the user'sfamily doctors, or manually inputted data related to the user's healthby the user's caretakers). The historical health data of a userdescribes a global view of the user's lifestyle and wellness. Forexample, the historical health data may include information about theuser's vision, such as the user's eyeglass or contact prescription,whether the user has cataracts, glaucoma, or other eye-relatedconditions, results of annual vision examinations by a doctor, amongother types of data. The historical health data could also includeinformation about any conditions the user has that could affect the riskof falls for that individual, such as Parkinson's disease, factors inthe user's medical history that might affect the user's balance (e.g.,ear infections or other ear conditions, posture or back issues such asscoliosis, problems with legs, feet, or knees such as a prior kneeinjury, among other data).

The fall risk module 110 processes user inputs, data from the wearablesensor 140, data from the depth sensor 150, data from the externalsource 160, and/or data from a local database, and trains a machinelearning model to determine a fall risk score for the user. The userinputs are provided by the user via the client device 130 (or in somecases via the wearable device holding the wearable sensor, such as awristwatch) and include information such as their age, history ofprevious falls, vision, any use of assistive devices, lightingconditions, history of neurological disorders, and the like. In oneembodiment, based on the determined fall risk score, the fall riskmodule 110 provides information to the user or a person associated withthe user such as the user's caretaker, physician, family and friends,and the like. The information may include a prediction of the user'sfall risk and/or a recommendation regarding how the user can reduce hisor her risk of falling. The fall risk module 110 is further describedbelow and with reference to FIGS. 2-6.

Fall Risk Module

FIG. 2 is a block diagram of a fall risk module 110 of an adaptivesystem to automatically quantify fall risk, according to one embodiment.The fall risk module 110 assesses the fall risk of the user andgenerates recommendations regarding ways that the user can reduce his orher risk of falling. In the embodiment illustrated in FIG. 2, the fallrisk module 110 has a machine learning module 200, fall risk model 210,fall risk prediction module 220, wearable sensor data processing module230, depth sensor data processing module 240, structured test module250, user interface module 260, external source store 280, and user datastore 285. In alternative configurations, different, fewer, and/oradditional components may be included in the fall risk module 110. Forexample, the fall risk module may integrate with various third partyhardware or software to provide a comprehensive solution to users of thefall risk module 110. Functionality of one or more of the components maybe distributed among the components in a different manner than isdescribed herein.

The machine learning module 200 uses machine learning techniques togenerate the fall risk model 210. In an embodiment, the machine learningmodule 200 may generate a model based on optimization of different typesof fall risk scoring models, including but not limited to algorithmsthat analyze processed and/or raw data from each wearable sensorseparately, or in any combination with processed data from the depthsensor, data from user inputs, or data from structured tests. Forexample, the machine learning module 200 may generate a classifier thattakes as input a gait profile measured by motion sensors for a user andreturns true if the determined fall risk based on the gait profile isgreater than that of the average population and false otherwise. Thefall risk prediction model 220 can use the output of the generatedclassifier to generate a fall risk score and/or predicted time periodduring which the user is more vulnerable to experience a fall. Otherembodiments can use other machine learning techniques for determiningfall risk, for example, tree-based models, kernel methods, convolutionalneural networks, splines, or an ensemble of one or more of thesetechniques.

In some embodiments, the fall risk module 110 may use multiple fall riskmodels 210 for determining the fall risk for users. For example, themachine learning module 200 may divide the set of users of the fall riskmodule 100 into different subsets of users and generate one fall riskmodel 210 for each subset of user. The subsets of users may bedetermined based on demographic information describing the users, forexample, age, gender, ethnic background, etc. Since different sets ofusers may have different characteristics determining their fall risk, afall risk model 210 for each set of users is likely to provide moreaccurate score compared to a single fall risk model 210 for the entireset of users.

In an embodiment, the features used for a model for a set of users maydepend on the value of the attribute that characterizes the set. Forexample, if the frequency of previous falls is used as an attribute thatcharacterizes the set, different sets may be determined for users in agroup that frequently experiences falls and users in a group that havenever experienced a fall. The features used for the model for each setmay depend on the frequency of falls for the set. For example, the modelfor a set of users who typically experience a fall once a month may usethe times of day during which the users' previous falls occurred as afeature. In another different example, a model for a set of users whohave never fallen may use the users' gait profiles as a feature.

The fall risk prediction model 220 generates and provides a predictionof a user's fall risk to the user interface manager 260 for presentationto the user via the client device 130. A person associated with the user(e.g., the user's caregivers and other health care providers, family andfriends, and the like) may also view the fall risk on the client device.In addition to the fall risk, the fall risk prediction model 220 alsogenerates feedback, using information from the external source store 280and user data store 285, for presentation to the user, includingrecommendations regarding how the user can reduce their risk of falling,and how the user's fall risk compares with those of the population as awhole. For example, if the user has poor posture that is tied to agreater risk of falling, the fall risk prediction model 220 may notifythe user and/or another clinician that the user's fall risk is greaterthan average and suggest that the user use a walking cane to reducetheir fall risk. Other examples of recommendations include suggestionsto change conditions in a user's environment, such as adjusting poorlighting conditions to provide more light to the user to avoid trips andfalls, removal of items on the floor that present tripping hazards,adjustment of furniture in the room to avoid blocking common walkwaysand avoid bumping into furniture, providing better lighting at stairs ora step between two levels in a home, and so forth. Further examplesincluding suggestions to a user to change posture or gait, such asfocusing on standing up straighter, providing back strengtheningexercises to improve posture, suggesting wearing of a back brace or kneebrace to address the source of poor posture, recommending walking sloweror taking smaller steps, recommending more swinging of your arms toimprove balance, suggesting particular canes or walkers to address theperson's particular issue with their gait, etc. A person associated withthe user may also be notified of the user's fall risk and correspondingrecommendations.

The wearable sensor data processing module 230 takes as input the datacollected by the wearable sensor 140 and processes them to generatefeatures used by the machine learning module 200. For example, 3-axis(e.g., x, y, and z) data from the inertial measurement unit (IMU) in thewearable sensor 140 can be processed to generate a gait profile of theuser. Further, the IMU data can also be processed to identify when theuser has experienced a fall. In another example, data from a photo diodein the wearable sensor 140 can be processed to generate the heart rateand pulse of the user. The gait profile, events such as a fall, heartrate, pulse, and other information associated with the user generated bythe wearable sensor data processing module 230 is stored in the userdata store 285.

The depth sensor data processing module 240 takes as input the datacollected by the depth sensor 150 and processes them (e.g., using videoprocessing techniques known to one skilled in the art) to generatefeatures used by the machine learning module 200. For example, arecording of the user in a standing position can be processed (e.g.,using skeletal tracking algorithms) to generate a model of the user'sposture and determine the user's center of gravity. Further, the depthsensor data can also be processed to identify when the user changes froma sitting position to a standing position. The user's posture, center ofgravity, transition from sitting to standing positions, and otherinformation associated with the user generated by the depth sensor dataprocessing module 240 is stored in the user data store 285.

The structured test module 250 prompts the user to complete a structuredtest, for example, to measure the trace of any swaying motions while theuser is standing or to improve posture through regular strengthtraining. The structured test module 250 provides instructions regardingthe structured test to the user interface manager 260 for presentationto the user via the client device 130 or via a wearable device thatholds the wearable sensor 140. The prompt occurs at a predeterminedfrequency (e.g., daily and weekly) and the user may be prompted tocomplete different structured tests at varying frequencies. The datarecorded by the wearable sensor 140 and depth sensor 150 while the useris completing the structured test is processed by the wearable sensordata processing module 230 and depth sensor data processing module 240,and are used as features to train the fall risk model 210 and to updatefall risk predictions. Data recorded during the structured test can bestored in the user data store 285. The structured tests may vary dailyand may be presented to the user one at a time or in a set of one ormore tests. Further, the structured tests selected for users, and theparameters of the selected tests, may be customized to each user basedon information associated to each particular user. The structured testmodule 250 may provide live feedback to the user during and/or aftercompleting the test. Information from feedback from the user duringand/or after completing the test may be used to train the fall riskmodel 210.

In one embodiment of the structured test, the structured test module 250sends a text message notification via a smart phone client device 130each morning to a user, who is wearing a wristwatch with a wearablesensor 140, requesting that the user complete a mean radius of tracetest. The user opens a software application on the smart phone to beginthe test. The user also has the option to postpone the test via thesoftware application. The software application receives audibleinstructions and visual instructions from the structured test module 250and provides audible instructions via the speakers of the smart phoneand visual instructions via the user interface of the smart phone to theuser, guiding the user through the steps of the test. For the first stepof the test, the structured test module 250 instructs the user to standup, and the wearable sensor 140 collects data indicating the user'sstarting position. For the second step, the structured test module 250instructs the user to remain standing for a continuous ten secondperiod. A count down timer is displayed on the smart phone indicatingthe remaining duration of the ten second period. Once the ten secondperiod has elapsed, the smart phone provides a visual message and audiomessage indicating that the test is complete. Data collected from thewearable sensor 140 during the test is used to calculate a mean radiusof trace, which indicates how much the user swayed while standing. Ifdata processed from previous tests completed by the user indicate thatthe user has a low mean radius of trace, which suggests that the userhas strong postural stability, the user may be prompted less frequentlyto complete this test in the future. On the other hand, if dataprocessed from previous tests completed by the user indicate that theuser has a large mean radius of trace, which suggests that the user hasweak postural stability, the ten second period may be adjusted to ashorter or longer time period of time in future tests.

In another embodiment of the structured test, the structured test module250 measures the heart rate and/or pulse of the user while the user issitting down or in a recumbent position and again immediately after theuser has stood up. Heart rate and pulse data may be collected by aninfrared-light based sensor included in the wearable sensor 140. Thefall risk module 110 calculates the difference between the heart rateand/or pulse measurements before and after the user has stood up. Alarge calculated difference can suggest that the user may have posturalhypotension, which is associated with a low blood pressure drop thatoccurs when an individual stands up from a sitting or recumbentposition. After the user stands up during this test, the structured testmodule 250 may immediately prompt the user via the client device 130 toindicate whether the user feels lightheaded, which is a common symptomof individuals with postural hypotension when they stand up.

The external source store 280 stores information from the externalsource 160 including, but not limited to, data from a population of bothhealthy and unhealthy users, as well as medical practice standards. Datafrom a population of both healthy and unhealthy users may includestatistics regarding the mean and standard deviation of the number offalls per week for both groups of users categorized as healthy andunhealthy, among other data. The medical practice standards may includerecommending an increase in fluid and dietary salt intake to users whohave postural hypotension as an identified risk factor. Other examplesof medical practice standards stored in the source 160 includerecommending physical balance therapy for users who suffer from posturalinstability, and advising to avoid interactions between differentsedative medications for users who demonstrate chronic use of sedativesfrom their medical history, among other data.

Mean Radius of Trace Detection

FIG. 3A is a chart 300 a illustrating the trace of a healthy user with afall risk near average as measured by the adaptive system toautomatically quantify fall risk, according to one embodiment. The trace330 a represents any swaying motions of the healthy user while in astanding position and is mapped on a 2D plane parallel to the groundwith an x-axis 320 and y-axis 310. In alternate embodiments, the trace330 a can be mapped to combinations of different axis in a 2D plane,mapped in 3D, or mapped in a 1D line. The trace 330 a can be generatedusing data processed by the wearable sensor data processing module 230and/or depth sensor data processing module 240 collected by the wearablesensor 140 and/or the depth sensor 150. This information may becollected for the user during a structured test where the user is askedto stand in one position and the sway of the user is measured for aperiod of time, or this information may be collected throughout the dayas the user stands or moves. The mean radius of trace from the origin of2D plane can be calculated by the fall risk module 110 based on thetrace 330 a and averaging the distance that the user deviated from astarting position, in the case of a structured test, or from adetermined average or reference position, in cases outside of astructured test, over a period of time. The mean radius of trace iscompared against the mean radius of trace for a larger population ofusers and/or that of an unhealthy user. For a healthy user, the trace330 a is expected to show only small motions in the 2D plane.

FIG. 3B is a chart illustrating the trace of an unhealthy user with afall risk greater than average as measured by the adaptive system toautomatically quantify fall risk, according to one embodiment. The trace330 b represents any swaying motions of the unhealthy user while in astanding position and is mapped on a 2D plane parallel to the groundwith an x-axis 320 and y-axis 310. In alternate embodiments, the trace330 b can be mapped to combinations of different axis in a 2D plane,mapped in 3D, or mapped in a 1D line. The trace 330 a can be generatedusing data processed by the wearable sensor data processing module 230and/or depth sensor data processing module 240 collected by the wearablesensor 140 and/or the depth sensor 150. The mean radius of the tracefrom the origin of 2D plane can be calculated by the fall risk module110 based on the trace 330 a and averaging the distance that the userdeviated from a starting position, in the case of a structured test, orfrom a determined average or reference position, in cases outside of astructured test, over a period of time. The mean radius of trace iscompared against the mean radius of the trace for a larger population ofusers and/or that of a healthy user. For an unhealthy user, the trace330 a is expected to show larger motions, relative to those of a healthyuser, in the 2D plane. For example, unhealthy users with multiplesclerosis typically have symptoms of poor balance, which would result inlarger swaying motions while standing and thus, a greater mean radius ofthe trace. The machine learning module 200 can use data from the tracesof the unhealthy and the healthy user to train the fall risk model 210,which may influence the generated fall risk scores.

In other embodiments, the trace 330 a and the trace 330 b can representthe anterior, posterior, medial, and lateral coordinates of the user'sbody while attempting to stand still. The 2D plane may correspond to thesagittal plane (i.e., the plane parallel to the sagittal suture of thehuman body), the frontal/coronal plane (i.e., the plane that divides thehuman body into the back and front, or posterior and anterior,sections), and the transverse plane (i.e., the plane that divides thehuman body into the cranial and caudal, or head and tail, sections). Thetrace 330 a and the trace 330 b may be mapped in 3D onto any combinationof two or more of these three 2D planes. The mean radius of trace can becalculated in any combination of one, two, or three dimensions. Thechart 300 a and the chart 300 b may also show the pitch, roll, and yawangles exhibited as the user is swaying, which can be demonstrative ofbalance difficulties or disorders. The machine learning module 200 canuse these three angles from the user to train the fall risk model 210,which may influence the generated fall risk scores.

Center of Gravity Detection

FIG. 4A is a diagram illustrating the posture and center of gravity of ahealthy user with a fall risk near average as measured by the adaptivesystem to automatically quantify fall, risk according to one embodiment.The healthy user's standing posture 430 a is mapped on a 2D planeorthogonal to the ground with an x-axis 420 and z-axis 410. In alternateembodiments, the posture 430 a can be mapped to combinations ofdifferent axis in a 2D plane, mapped in 3D, or mapped in a 1D line. Theposture 430 a can be generated using data processed by the wearablesensor data processing module 230 and/or depth sensor data processingmodule 240 collected by the wearable sensor 140 and/or the depth sensor150. The healthy user's center of gravity 440 a can be calculated by thefall risk module 110 based on the posture 430 a and compared against thecenter of gravity of a standing posture for a larger population of usersand/or that of an unhealthy user. For a healthy user, the center ofgravity 440 a is expected to be close to the z-axis 410 because the userwill be standing upright. The center of gravity information may becollected during a structured test performed by a user or may becollected throughout the day.

FIG. 4B is a diagram illustrating the posture and center of gravity ofan unhealthy user with a fall risk greater than average as measured bythe adaptive system to automatically quantify fall risk, according toone embodiment. The unhealthy user's standing posture 430 b is mapped ona 2D plane orthogonal to the ground with an x-axis 420 and z-axis 410.In alternate embodiments, the posture 430 b can be mapped tocombinations of different axis in a 2D plane, mapped in 3D, or mapped ina 1D line. The posture 430 b can be generated using data processed bythe wearable sensor data processing module 230 and/or depth sensor dataprocessing module 240 collected by the wearable sensor 140 and/or thedepth sensor 150. The unhealthy user's center of gravity 440 b can becalculated by the fall risk module 110 based on the posture 430 b andcompared against the center of gravity of a standing posture for alarger population of users and/or that of a healthy user. For anunhealthy user, the center of gravity 440 b is expected to be furtherfrom the z-axis 410 because the user may not be standing upright. Forexample, unhealthy users with Parkinson's have symptoms of posturalinstability, which typically results in a forward or backward lean withthe drooped shoulders and a bowed head. Since the postural instabilitycan cause the unhealthy user to experience more falls, the determinedcenter of gravity 440 b can be used by the fall risk module 110 togenerate a greater fall risk score for the unhealthy user.

Machine Learning Module

FIG. 5A shows a diagram illustrating timelines with fall events of auser, according to one embodiment. In the embodiment illustrated in FIG.5A, there are three timelines each representing a different day duringwhich the fall risk module 110 receives data from a user. The days maybe subsequent days or days separated by more than one day. In day 1 502,the fall risk module 110 does not receive data indicating that the userhas experienced a fall event. In day 2 504, the fall risk module 110receives data indicating that the user has experienced a fall event A514, represented by the cross mark on the timeline at time t₂ 510. Thefall horizon A 512 represents the period of time immediately precedingthe fall event A 514. The fall horizon A 512 is a predetermined durationof time from time t₁ 508 to time t₂ 510. In day 3 506, the fall riskmodule 110 receives data indicating that the user has experienced a fallevent B 530, represented by the cross mark on the timeline at time t₈526. Fall event B 530 corresponds with fall horizon B 528, which startsat time t₅ 520. In day 3 506, window A 532 is the period of timestarting from time t₃ 516 and ending at time t₄ 518, and window B 534 isthe period of time starting from time t₆ 522 and ending at time t₇ 524.Window A 532 and window B 534 may be the same or different duration oftime (e.g., both windows can be 30 seconds in duration), and in otherembodiments, different windows may overlap in a timeline. In theembodiment illustrated in FIG. 5A, Window A 532 does not fall withinfall horizon B 528, while window B 534 does fall within fall horizon B528. Data collected from sources such as the client device 130, wearablesensors 140, depth sensors 150, and/or external sources 160 duringwindow A 532, window B 534, and additional windows representing periodsof time in the timeline of a day may be used by the fall risk module 110to generate a tensor, which is further described in FIG. 5B, to trainthe fall risk model 210, which may influence the generated fall riskscores.

FIG. 5B shows a diagram illustrating a tensor 560 of data used by a fallrisk module 110, according to one embodiment. In the embodimentillustrated in FIG. 5B, the tensor 560 is represented by a matrix withthree dimensions. The first dimension, window 562, represents one ormore windows (i.e., periods of time) during which the fall risk model210 received data corresponding to a user. The second dimension, rawsample 564, represents one or more raw data samples during which thefall risk model 210 received data corresponding to a user. The thirddimension, data source 566, represents one or more sources of data fromwhich the fall risk model 210 received data corresponding to a user. Thesources of data may include devices such as the client device 130,wearable sensors 140, and depth sensor 150, as well as other sourcessuch as external sources 160. Further, the source of data may be aspecific channel of data from a device, for instance, the x-axis channelof a 3-axis accelerometer and/or gyroscope, or any combination ofchannels of data from a device, for instance, the roll, pitch, and yawangles from an inertial measurement unit. The data element 568represents one data element in the tensor 560. In one example, the dataelement 568 may be collected from the first window of data in the window562 dimension, the last raw sample in the raw sample 564 dimension, andthe data source that corresponds to the x-axis channel of anaccelerometer in the data source 566 dimension. The value of the dataelement may be a numerical value, clear text, Boolean, or otherrepresentations of data. Following in the same example above, the dataelement from an x-axis channel of an accelerometer may be a numericalvalue from 0 to 255. One or more tensors 560 may be used to determinefeatures, further described in FIG. 5C, to train the fall risk model210.

In other embodiments, the tensor 560 may have fewer, more, and/ordifferent dimensions. For example, instead of the raw sample 564dimension, the tensor 560 may have a dimension representing one or moreprocessed data samples during which the fall risk model 210 receiveddata corresponding to the user. In another example, instead of thewindow 562 dimension, the tensor 560 may have a dimension representingone or more structured tests completed by the user during which datafrom the user was received by the fall risk model 210.

FIG. 5C shows a data flow diagram 580 illustrating the interactionsbetween various types of data used to determine a fall risk score,according to one embodiment. As shown in FIG. 5, the fall risk model 210trained by the machine learning module 200 is used to determine a fallrisk score 594 for the user. The machine learning module 200 may extractfeatures 592 from data including wearable_sensor_data 510 from thewearable sensors 140, depth_sensor_data 520 from the depth sensor 150,external_source_data 530 from the external source 160 and/or externalsource store 280, user_data 540 from the client device 130 and/or userdata store 285, and population_data 550 from the external source 160 touse as training data sets to train the fall risk model 210. The machinelearning module 200 may also extract features 592 from the tensor 560.Further, an expert may manually provide features and training data setsto the fall risk model 210 (e.g., through the client device 130). Forexample, a data scientist may discover features or motifs that areassociated with fall events.

In one instance, referring back to day 3 506 illustrated in FIG. 5A, thedata scientist discovers from the data of a user collected by wearablesensors 140 that the user was eating a meal during window B 534, but theuser did not take prescribed medication recommended to be taken with themeal. Since the user experienced fall event B 530 shortly following themeal during window B 534, the data scientist may determine that eating ameal without taking prescribed medication may be a feature that predictsa higher risk of a fall event occurring. After this feature is providedto the fall risk module 110, the machine learning module 200 may performa retrospective identification of fall events by searching foroccurrences of this feature in previously collected data (e.g., datafrom day 2 504 and day 1 502). Results of the retrospectiveidentification may either reinforce or weaken the feature for trainingthe fall risk model 210 depending on how strongly the feature iscorrelated with occurrences of fall events.

In another instance, once the fall risk model 210 has been trained withan initial training data set, the machine learning module 200 usesmachine learning techniques known to one skilled in the art such as deeplearning convolutional neural networks to identify additional featuresand motifs used to train the fall risk model 210 and predict fall risk.For example, using a neural network, the machine learning module 200 mayidentify a particular pattern from data in the tensor 560 involving acombination of data from multiple accelerometer and gyroscope axischannels that often precedes a fall event. The features and motifsidentified by the machine learning module 200 or an expert (e.g., a datascientist) may be used by the fall risk module 110 to generate a machinelearning classifier to predict a user's fall risk.

In one embodiment, a classifier generated by the fall risk module 110predicts whether each window of collected data in the tensor 560 fallswithin a fall horizon or does not fall within a fall horizon. Forexample, referring back to day 3 506 illustrated in FIG. 5A, predictingthat window B 534 falls within a fall horizon would be a correctprediction because window B 534 falls within fall horizon B 528. Theclassifier, which may be used by the fall risk model 210 and/or fallrisk prediction module 220, determines a score based on an input windowof a tensor 560 and compares the score against a threshold value. If thescore is above the threshold, the classifier predicts that the windowfalls within a fall horizon; otherwise, the classifier predicts that thewindow does not fall within a fall horizon. The threshold may beadjusted over time based on the accuracy of the classifier as determinedusing retrospective analysis or manual input from an expert. Forexample, if the classifier is causing too many false positives (e.g.,incorrect predictions of windows falling within a fall horizon), thefall risk module 110 may increase the threshold value. On the otherhand, if the classifier is causing too many false negatives (e.g.,incorrect predictions of windows not falling within a fall horizon), thefall risk module 110 may decrease the threshold value.

FIG. 6 is a flowchart of the process 600 of periodically retraining amodel for determining a fall risk score for a user, according to oneembodiment. The fall risk module 110 receives/collects and processes 610data from the wearable sensors 140, depth sensor 150, external source160, user data, and population data. The machine learning module 200generates 620 a machine learning model (e.g., the fall risk model 210)for determining fall risk scores corresponding to each subset of users.The generation 620 of the fall risk model 210 may comprise training thefall risk model 210 based on a training data set and/or a tensor 560. Anexpert may analyze various the features 592 and determine whichparticular features correlate to a greater or lower fall risk. Forexample, a high mean radius of trace that indicates a user demonstrateslarge swaying motions while standing may be correlated to a greater fallrisk. In another example, a center of gravity in standing posture thatindicates a user leaning forward may also be correlated to a greaterfall risk. On the other hand, a lower fall risk may be correlated to auser who has a gait profile similar to the average gait profile of ahealthy user in the general population. Based on the trained fall riskmodel 210 and data from one or more tensors 560, the fall risk module110 determines 630 a fall risk score (e.g., fall risk score 594) for theuser.

If the determined fall risk score exceeds a threshold value, then thefall risk module 110 notifies 640 a combination of the health careproviders, caregivers, family, and friends of the user, as well as anyother person associated with the user regarding the score. Thenotification may be in the form of an automatically generated electronicmessage (e.g., text message and email) sent to the client device 130.The threshold value may be set by the expert, based on the externalsource 160, based on the tensor 560 processed by the machine learningmodule 200, or based on another person or source of information.

The machine learning module 200 periodically retrains 650 the modelcorresponding to each subset of users. The machine learning module 200may retrain each model at a fixed frequency. The model corresponding toeach subset may be retrained 650 at a different frequency compared toother models. The machine learning module 200 may use informationdescribing how users of each subset interact with the fall risk module110 via the client device 130, or data from the tensor 560, to determinea rate at which the corresponding model is retrained 650. After themodel is retrained 650, the fall risk module 110 again determines 630 afall risk score (e.g., fall risk score 594) for the user, which may bedifferent than the previously determined score because the model hasbeen updated.

Alternative Applications

The features and advantages described in the specification are not allinclusive and, in particular, many additional features and advantageswill be apparent to one of ordinary skill in the art in view of thedrawings, specification, and claims. Moreover, it should be noted thatthe language used in the specification has been principally selected forreadability and instructional purposes, and may not have been selectedto delineate or circumscribe the inventive subject matter.

The foregoing description of the embodiments of the invention has beenpresented for the purpose of illustration; it is not intended to beexhaustive or to limit the invention to the precise forms disclosed.Persons skilled in the relevant art can appreciate that manymodifications and variations are possible in light of the abovedisclosure.

Some portions of this description describe the embodiments of theinvention in terms of algorithms and symbolic representations ofoperations on information. These algorithmic descriptions andrepresentations are commonly used by those skilled in the dataprocessing arts to convey the substance of their work effectively toothers skilled in the art. These operations, while describedfunctionally, computationally, or logically, are understood to beimplemented by computer programs or equivalent electrical circuits,microcode, or the like. Furthermore, it has also proven convenient attimes, to refer to these arrangements of operations as modules, withoutloss of generality. The described operations and their associatedmodules may be embodied in software, firmware, hardware, or anycombinations thereof.

Any of the steps, operations, or processes described herein may beperformed or implemented with one or more hardware or software modules,alone or in combination with other devices. In one embodiment, asoftware module is implemented with a computer program productcomprising a computer-readable medium containing computer program code,which can be executed by a computer processor for performing any or allof the steps, operations, or processes described.

Embodiments of the invention may also relate to an apparatus forperforming the operations herein. This apparatus may be speciallyconstructed for the required purposes, and/or it may comprise ageneral-purpose computing device selectively activated or reconfiguredby a computer program stored in the computer. Such a computer programmay be stored in a tangible computer readable storage medium or any typeof media suitable for storing electronic instructions, and coupled to acomputer system bus. Furthermore, any computing systems referred to inthe specification may include a single processor or may be architecturesemploying multiple processor designs for increased computing capability.

Finally, the language used in the specification has been principallyselected for readability and instructional purposes, and it may not havebeen selected to delineate or circumscribe the inventive subject matter.It is therefore intended that the scope of the invention be limited notby this detailed description, but rather by any claims that issue on anapplication based hereon. Accordingly, the disclosure of the embodimentsof the invention is intended to be illustrative, but not limiting, ofthe scope of the invention, which is set forth in the following claims.

What is claimed is:
 1. A computer-implemented method comprising:conducting a structured test with a user, the structured test includingpresenting an instruction to the user to perform at least one action fora test duration to test a current state of the user; receiving, from oneor more sensors worn by the user or placed near the user, sensor datathat indicates at least a motion of the user or a posture of the user;measuring, with the sensor data, the motion of the user or the postureof the user during the structured test; adjusting at least one of atiming of providing an instruction to the user to perform at least oneaction for a subsequent structured test and a test duration of thesubsequent structured test based on the measured motion of the user orthe measured posture of the user; and conducting the subsequentstructured test with the user based on the adjusted at least one of thetiming and the test duration.
 2. The computer-implemented method ofclaim 1, wherein the structured test comprises a test for measuring amean radius of trace of the user while standing.
 3. Thecomputer-implemented method of claim 2, wherein presenting theinstruction to the user to perform the at least one action to test thecurrent state of the user includes providing an instruction to the userto stand up.
 4. The computer-implemented method of claim 3, wherein themeasuring the motion of the user or the posture of the user during thestructured test includes measuring, with the sensor data, a startingposition of the user and the mean radius of trace of the user while theuser is standing, wherein the mean radius of trace represents an averagedistance that the user deviated from the starting position over the testduration.
 5. The computer-implemented method of claim 3, whereinmeasuring the motion of the user or the posture of the user during thestructured test includes measuring, with the sensor data, at least oneof a pitch, a roll, and a yaw angle of the user during the structuredtest.
 6. The computer-implemented method of claim 1, further comprisingproviding features based on the sensor data to a classifier trained ondata about a plurality of users to generate a score indicating alikelihood of whether an input window is within a fall horizon, whereinthe fall horizon is a predetermined period of time that immediatelyprecedes a fall event.
 7. The computer-implemented method of claim 6,further comprising, in response to determining that the score is greaterthan a predetermined threshold, notifying a client device withinformation about a prediction of a fall risk of the user.
 8. Thecomputer-implemented method of claim 6, wherein the features based onthe sensor data include a measured mean radius of trace of the userwhile standing.
 9. The computer-implemented method of claim 1, whereinthe structured test comprises detecting a center of gravity of the userwhile standing.
 10. A system, comprising: one or more sensors; and acomputing device having logic stored thereon that, in response toexecution by the computing device, causes the computing device toperform actions comprising: presenting an instruction to a user toperform at least one action of a structured test for a test duration totest a current state of the user; receiving, from the one or moresensors, sensor data that indicates at least a motion of the user or aposture of the user during the structured test; measuring, using thesensor data, the motion of the user or the posture of the user duringthe structured test; adjusting at least one of a timing of presenting aninstruction to the user to perform at least one action for a subsequentstructured test and a test duration of the subsequent structured testbased on the measured motion of the user or the measured posture of theuser; and presenting the instruction to the user to perform at least oneaction of the subsequent structured test based on the adjusted at leastone of the timing and the test duration.
 11. The system of claim 10,wherein the one or more sensors include at least one sensor worn by theuser or placed near the user.
 12. The system of claim 11, wherein thecomputing device is worn by the user, and at least one sensor of the oneor more sensors is incorporated into the computing device.
 13. Thesystem of claim 10, wherein the structured test comprises a test formeasuring a mean radius of trace of the user while standing.
 14. Thesystem of claim 13, wherein presenting the instruction to the user toperform the at least one action to test the current state of the userincludes providing an instruction to the user to stand up.
 15. Thesystem of claim 14, wherein the measuring the motion of the user or theposture of the user during the structured test includes measuring, withthe sensor data, a starting position of the user and the mean radius oftrace of the user while the user is standing, wherein the mean radius oftrace represents an average distance that the user deviated from thestarting position over the test duration.
 16. The system of claim 14,wherein measuring the motion of the user or the posture of the userduring the structured test includes measuring, with the sensor data, atleast one of a pitch, a roll, and a yaw angle of the user during thestructured test.
 17. The system of claim 10, wherein the actions furthercomprise providing features based on the sensor data to a classifiertrained on data about a plurality of users to generate a scoreindicating a likelihood of whether an input window is within a fallhorizon, wherein the fall horizon is a predetermined period of time thatimmediately precedes a fall event.
 18. The system of claim 17, whereinthe actions further comprise, in response to determining that the scoreis greater than a predetermined threshold, notifying a client devicewith information about a prediction of a fall risk of the user.
 19. Thesystem of claim 17, wherein the features based on the sensor datainclude a measured mean radius of trace of the user while standing. 20.The system of claim 10, wherein the structured test comprises detectinga center of gravity of the user while standing.